Night vision imaging systems produce visible images of an environment having minimal ambient light, which would otherwise not be visible to the human eye. Such systems are used by military and law enforcement units, as well as various civilian applications. One such application is for improving the visibility of a vehicle driver during night, rain, fog, or other poor visibility driving conditions. The generated image of the area surrounding the vehicle may be processed to provide various driver assistance and safety features, such as: forward collision warning (FCW), lane departure warning (LDW), traffic sign recognition (TSR), and the detection of relevant entities such as pedestrians or oncoming vehicles. The image may also be displayed to the driver, for example projected on a head-up display (HUD) on the vehicle windshield. A vehicle night vision system may also be used to enable autonomous driving at low light levels or poor visibility conditions.
An imaging system may be passive or active. Some passive imaging systems operate by amplifying the existing low level ambient light in the environment to produce a visible video image (i.e., a reflected image). Other passive imaging systems are based on a thermal or infrared camera, which senses differences in infrared radiation emitted by objects in the surrounding area, and generates a video image according to the sensed radiation differences (i.e., an emitted image). An active imaging system involves transmitting light from a light source to illuminate the environment, and accumulating the reflected light by an imaging sensor, providing a visible image even when there is virtually no existing ambient light in the environment. The light source, which may be, for example, an LED, a filtered light bulb, or a laser, may transmit the light in the form of continuous wave (CW) or in a series of pulses.
Contemporary image sensors are typically semiconductor based, such as charge-coupled devices (CCD), or active-pixel sensors (APS) produced using the complementary metal-oxide-semiconductor (CMOS) or the N-type metal-oxide-semiconductor (NMOS) processes. Examples of such image sensors include: Intensified-CCD (ICCD)/Intensified-CMOS (ICMOS); Electron Multiplying CCD (EMCCD); Electron Bombarded CMOS (EBCMOS); Hybrid FPA (CCD or CMOS, such as InGaAs, HgCdTe); and Avalanche Photo-Diode (APD) focal plane array. In a CMOS sensor, each array pixel is associated with respective electronic components and circuitry for converting the incident light into a corresponding electrical charge, including a photodetector, an active amplifier, a capacitor, and switching components to readout the photodetector integrated charge. Each pixel group in the sensor array may be independently controlled to collect the incident radiation and integrate the charge for a selected exposure, where the pixel data undergoes readout after an entire frame has been captured. In comparison to camera tube sensors (e.g., photomultiplier, image intensifier based sensors, or light multipliers), CCD/CMOS image sensors are generally lower cost to produce and can function satisfactorily in a variety of environments and temperature conditions (e.g., daylight operation), as they are less susceptible to saturation, blooming effects and loss of sensitivity. An additional drawback of camera tube sensors is the “screen burn-in” effect, caused by the inability of the intensifier to handle a constant high level input of photons. Typically, when an image intensifier is activated in daytime (without optical or electrical protection) the tube would burn out in minutes. Likewise, a constant bright object may also cause a burn area (spot) in the image intensifier.
The technique of synchronizing the illumination pulses with the camera activation in active imaging systems in order to image a particular depth of field, also known as “gated imaging”, is known in the art. This technique is disclosed, for example, in U.S. Pat. No. 7,379,164 to Inbar et al., entitled “Laser gated camera imaging system and method”; in U.S. Pat. No. 7,733,464 to David et al., entitled “Vehicle mounted night vision imaging system and method”; in U.S. Pat. No. 8,194,126 to David et al., entitled “Gated imaging”; in PCT Patent Application Publication No. WO2013/157001 to Grauer et al., entitled “Multiple gated pixel per readout”; and in PCT Patent Application Publication No. WO2013/179280 to Grauer et al., entitled “Gated imaging using an adaptive depth of field”. After the illumination pulse is transmitted, the camera remains in an off state (i.e., does not accumulate any reflected photons), while the pulse reaches the target area and light is reflected back toward the camera. When the reflected light is due to arrive at the camera, the camera is activated to open (i.e., accumulates reflected photons). After the pulse is received, the camera is turned back off, while awaiting the transmission and reflection of the subsequent illumination pulse. The camera remains off for the duration of time required for the pulse to travel toward the target area and be reflected back, and is subsequently activated only for the duration required to receive the reflected light from the desired depth of field. In this manner, the camera receives only reflections from the desired range, and avoids reflections from other objects, such as particles in the atmosphere which may cause backscattering and reduce the contrast of the target area in the generated image. Gated imaging may also be employed to diminish the potential for oversaturation and blooming effects in the sensor, by collecting fewer pulses from shorter distances, thereby lowering the overall exposure level of the camera to near-field scenery and avoiding high intensity reflections from very close objects. Similarly, the light intensity or the shape of the illumination pulse may be controlled as a function of the distance to the target object, ensuring that the intensity of the received reflected pulse is at a level that would not lead to overexposure of the image sensor.
Each image frame generally includes multiple gating cycles, where each cycle consists of a pulse transmission/reflection and a respective sensor exposure to collect the reflected pulse. The pulse width and exposure duration is determined as a function of various relevant parameters, including the desired depth of field to be imaged. During the period when the camera sensor is not exposed (i.e., while the light pulse may still be propagating through the atmosphere), the sensor ideally will not accumulate any photons. But in practice, a certain level of residual light may still enter the image sensor or be accumulated by the image sensor (i.e., signal charge can be stored in the memory node without being contaminated by parasitic light). This phenomenon of “leakage photons” is especially problematic in CMOS sensors, where it is difficult to mask the memory node (MN) and floating diffusion in the pixel level sensor (typical masking approaches include: micro-lens focusing light away from the MN, metal layers above the MN, potential attracting the photoelectrons to the photodiode, and potential barriers around the MN). Therefore, a relatively long non-exposure period (i.e., sensor “off” time) serves to increase the extent of accumulated residual photons, which increases the noise in the acquired image. Conversely, an exposure duration (sensor “on” time) that is too long will result in image degradation (“overexposure”). Therefore, minimizing the time period between the exposures of successive gating cycles would lead to a reduction of noise in the acquired image frame. A typical time-span of a single gating cycle for an image sensor in active imaging systems (e.g., the duration between successive pulse transmissions) is primarily a function of the imaging application, but is typically in the range of: hundreds of picoseconds (ps) to a few microseconds (μs).
The issue of residual photon accumulation during sensor non-exposure periods is particularly relevant to vehicle camera based imaging and especially for night vision systems, such as when facing the headlights of an oncoming vehicle, or more acutely when facing an oncoming vehicle with its own active imaging system (and associated light source), which would likely result in significant accumulation of these undesired light sources. Existing techniques for overcoming this issue primarily rely on spectral filters and high dynamic range (HDR) image sensors.
When there is relative motion between the camera and an imaged object (e.g., if the object is moving and/or the camera is moving), a single exposure image tends to result in blurring, as the moving object appears smeared across the image. Removing the motion blur effect via deconvolution is an ill-posed problem due to lost spatial frequencies in the original image. One approach for overcoming motion blurring involves rapidly opening and closing the camera shutter throughout the exposure duration in accordance with a binary pseudo-random sequence, as described in: Raskar, Ramesh, et al., “Coded Exposure Photography: Motion Deblurring using Fluttered Shutter”, ACM SIGGRAPH 2006 Papers, SIGGRAPH '06. (2006):795-804.